Papers with adaptive online sample selection approach
OASIS: Online Sample Selection for Continual Instruction Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for continual instruction tuning (CIT) use pre-trained reference models, which are impractical in CIT setups since future data are unknown. |
| Approach: | They propose an adaptive online sample selection approach that estimates each sample’s informativeness relative to all previously seen data and minimizes informative redundancy through iterative selection score updates. |
| Outcome: | Experiments on various large foundation models show that using only 25% of the data achieves comparable performance to full-data training and outperforms the state-of-the-art sampling methods. |